1. Field of the Invention
The present invention relates to an imaging apparatus that appropriately captures an image of a scene in which a main subject is dark and a scene with a large dynamic range, an imaging method used in the imaging apparatus, an integrated circuit that functions as the imaging apparatus, and a storage medium storing an imaging program used in the imaging apparatus.
2. Description of the Related Art
Imaging apparatuses, such as a digital still camera for capturing still images and a digital video camera for capturing moving images, capture images in the following manner. In an imaging apparatus, an optical system focuses light and forms an image through exposure control, and a solid-state image sensor, such as a CCD (charge-coupled device) image sensor or a CMOS (complementary metal oxide semiconductor) image sensor, converts the image to electric signals, which are analogue image signals. In the imaging apparatus, a circuit that performs signal processing, such as analogue front end processing, then processes the analogue image signals, and an A/D (analogue-to-digital) converter converts the analogue image signals to digital image data. The digital image data obtained by the imaging apparatus is subsequently subjected to image processing, such as video gamma correction (gamma correction with a gamma of 0.45), knee adjustment, luminance conversion, and color difference conversion. The processed digital image data is converted to data in a standardized format. More specifically, when the digital image data is still image data, the data is converted to, for example, JPEG (Joint Photographic Experts Group) data. When the digital image data is moving image data, the data is converted to, for example, MPEG (Moving Picture Experts Group) data or DV (Digital Video) data. The digital image data in the standardized format is recorded onto a recording medium, such as a memory card, a hard disk, an optical disc, or a magnetic tape.
The exposure light amount of the optical system (exposure light amount determined by the aperture value or the shutter speed) in the imaging apparatus (camera) is typically determined based on results of, for example, light amount detection units corresponding to a plurality of imaging areas into which the imaging surface is divided. When capturing an image of a backlit scene in which a main subject is dark, the imaging apparatus increases the luminance level of the captured image by setting a large aperture or setting a slow shutter speed and increasing the light exposure amount (increasing the amount of light entering the image sensor of the imaging apparatus). Also, the imaging apparatus performs processing to increase the luminance level of a captured image (backlit image) by converting the tones of the captured image using a tone curve (tone conversion characteristic) with which the luminance level of a dark region of the captured image (region with low tones) is increased.
When capturing an image of a backlit scene, which typically contains a dark backlit person as a main subject and a bright background such as sky, the imaging apparatus (camera) sets its exposure light amount in the manners (A) to (C) described below.
(A) The exposure light amount is set in a manner to prevent a sky region of a captured image from being overexposed and failing to reproduce color (to prevent the sky region from losing its tones) (the aperture is set smaller).
(B) The exposure light amount is set in a manner to expose a person region of a captured image with an optimum light amount (expose the person region to have appropriate tones (or luminance levels)) (the aperture is set larger).
(C) The exposure light amount is set as an intermediate value between the exposure light amounts set in the cases (A) and (B).
In the case (A), the captured image is in a severely backlit state. The person region of the captured image is extremely dark, whereas the sky region of the captured image is prevented from being overexposed and failing to reproduce color (the sky region of the captured image is prevented from losing its tones). In the case (B), the luminance level of the person region of the captured image is increased, whereas the sky region of the captured image is completely overexposed and fails to reproduce color. Both the images captured in the cases (A) and (B) are not favorable. To avoid such unfavorable images, the imaging apparatus usually uses the intermediate exposure light amount set in the case (C).
In the case (C), however, the sky region of the captured image is overexposed slightly, whereas the person region of the captured image is not sufficiently bright. The imaging apparatus fails to capture an image in which both the sky region and the person region have appropriate luminance levels in the case (C). In this manner, the imaging apparatus fails to capture an image with a satisfactory image quality even in the case (C).
The conventional imaging apparatus may also have a “backlight correction mode”, which permits the user to explicitly choose the case (B). The conventional imaging apparatus with the backlight correction mode can increase the exposure light amount to the exposure light amount set in the case (B) as instructed by the user. The imaging apparatus with the backlight correction mode also recognizes a face and uses the luminance level of the face as a reference when determining the degree by which the exposure light amount is increased. In the image captured using the backlight correction mode, the person would be bright but the background would be completely overexposed and fail to reproduce color. As a result, the image captured using the backlight correction mode may fail to have a high image quality.
The technique of increasing the exposure light amount is not the only technique of backlight correction. Another backlight correction technique is to increase the luminance level of a dark region of a captured image by converting the tones of the captured image using tone curves (tone conversion characteristic curves). Such a backlight correction technique will now be described with reference to FIG. 18.
FIG. 18 shows the characteristics of backlight correction performed using conventional tone curves (tone conversion characteristic curves). In FIG. 18, the horizontal axis indicates the input luminance level, whereas the vertical axis indicates the output luminance level. The backlight correction needs a tone curve (tone conversion characteristic curve) with which the luminance level of a dark region is increased as indicated by arrows in FIG. 18.
With a tone curve (tone conversion characteristic curve) X, the luminance level of a dark region of a captured image is increased intensively while the luminance level of a bright region of the captured image is being unchanged. However, the backlight correction is not performed in an appropriate manner through the tone conversion performed using the tone curve X. The image resulting from the tone conversion performed using the tone curve X would be an unnatural image with an extremely low contrast of only intermediate tones (for example, the tones of an image region indicated by a circle drawn with a dotted line in FIG. 18) as shown in FIG. 2 of Non-Patent Citation 1.
With a tone curve Y, the tones of the entire image decrease. The image resulting from the tone conversion performed using the tone curve Y would be less unnatural. Thus, the backlight correction can be performed through the tone conversion using the tone curve Y. However, the tone conversion using the tone curve Y increases the luminance level of not only a dark backlit region of the captured image but also a bright region of the image. The tone conversion using the tone curve Y lowers the contrast of, for example, a background sky region of the captured image. As a result, the background sky region may easily be overexposed and fail to reproduce color (the tones of a highlight region, such as a sky region, may easily be saturated). The tone conversion (backlight correction) using the tone curve Y is practically possible only when the contrast of the captured image is lowered to a permissible level and the highlight region of the captured image is overexposed and fails to reproduce color at a permissible level. In other words, the tone conversion (backlight correction) using the tone curve Y is possible only when such tone conversion does not correspond to drastic correction but corresponds to moderate correction. The tone conversion (backlight correction) using the tone curve Y is possible within a range in which unnaturalness of the resulting image is permissible. Therefore, the tone conversion (backlight correction) using the tone curve Y cannot be used to perform drastic backlight correction.
Also, when the tone conversion is performed using a curve (tone conversion characteristic curve) with which the luminance level of not only a dark region but also an image region with an intermediate luminance level is increased, the image resulting from the tone conversion would be less unnatural. However, a highlight region of the image resulting from such tone conversion would be too bright in the same manner as in the case (B). For example, a sky region of the image resulting from the tone conversion performed using such a curve (tone conversion characteristic curve) may be overexposed and fail to reproduce color (a sky region may lose its tones). Also, the image resulting from such tone conversion may fail to have a sufficiently high contrast across the entire image. As a result, even the tone conversion (backlight correction) performed using the curve (tone conversion characteristic curve) with which the luminance level of not only a dark region but also an image region with an intermediate luminance level is increased cannot be used to perform drastic backlight correction.
To solve the above problem, one technique uses different tone conversion characteristics according to positions across an image. For example, an imaging apparatus with such a technique partially (or locally) increases the luminance level of a dark region of an image, such as a backlit person region of an image, by processing the dark region using a tone conversion characteristic different from a tone conversion characteristic used to process a sky region of the image for example. Such a tone conversion (backlight correction) technique is known in the art as a visualization-characteristic-based technique (see FIGS. 3 to 5 of Non-Patent Citation 1, Patent Citation 1, and Patent Citation 2). Human eyes increase sensitivity when viewing a bright region and decrease sensitivity when viewing a dark region. The local backlight correction using such human eye's characteristics enables the contrast to be perceived by the human eyes, and enables visually natural processing to be performed. Such local backlight correction using the visualization-characteristic-based technique can therefore be used to perform drastic correction. Such local backlight correction enables the luminance level of an extremely dark person region of an image to be increased while maintaining the local contrast in the image.
Patent Citation 1 (International Publication No. WO 2005/027041) describes one such technique. More specifically, the patent document describes a technique for using a region surrounding a processing target pixel of an image. With this technique, for example, the histogram of a region surrounding a processing target pixel of an image is measured, and the tone curve (tone conversion characteristic curve) used for the target pixel is determined based on the distribution of values of the histogram. Alternatively, the average luminance level of the region surrounding the processing target pixel is calculated, and the tone curve (tone conversion characteristic curve) used for the target pixel is determined according to the luminance level of the region surrounding the processing target pixel.
Non-Patent Citation 1: “Contrast-Gain Based Visual Tone Mapping for Digital Photo Prints”, Yamashita et al., The Journal of Imaging Science and Technology, vol. 50, no. 5, pp. 458-468 (2006).
Patent Citation 1: International Publication No. WO 2005/027041
Patent Citation 2: International Publication No. WO 2005/027043